Prox-DBRO-VR: A Unified Analysis on Decentralized Byzantine-Resilient Composite Stochastic Optimization with Variance Reduction and Non-Asymptotic Convergence Rates

Decentralized Byzantine-resilient stochastic gradient algorithms resolve efficiently large-scale optimization problems in adverse conditions, such as malfunctioning agents, software bugs, and cyber attacks. This paper targets on handling a class of generic composite optimization problems over multi-agent cyberphysical systems (CPSs), with the existence of an unknown number of Byzantine agents. Based on the proximal mapping method, two variance-reduced (VR) techniques, and a norm-penalized approximation strategy, we propose a decentralized Byzantine-resilient and proximal-gradient algorithmic framework, dubbed Prox-DBRO-VR, which achieves an optimization and control goal using only local computations and communications. To reduce asymptotically the variance generated by evaluating the noisy stochastic gradients, we incorporate two localized variance-reduced techniques (SAGA and LSVRG) into Prox-DBRO-VR, to design Prox-DBRO-SAGA and Prox-DBRO-LSVRG. Via analyzing the contraction relationships among the gradient-learning error, robust consensus condition, and optimal gap, the theoretical result demonstrates that both Prox-DBRO-SAGA and Prox-DBRO-LSVRG, with a well-designed constant (resp., decaying) step-size, converge linearly (resp., sub-linearly) inside an error ball around the optimal solution to the optimization problem under standard assumptions. The trade-offs between the convergence accuracy and the number of Byzantine agents in both linear and sub-linear cases are characterized. In simulation, the effectiveness and practicability of the proposed algorithms are manifested via resolving a sparse machine-learning problem over multi-agent CPSs under various Byzantine attacks.Comment: 14 pages, 0 figure

On classification of singular matrix difference equations of mixed order

This paper is concerned with singular matrix difference equations of mixed order. The existence and uniqueness of initial value problems for these equations are derived, and then the classification of them is obtained with a similar classical Weyl's method by selecting a suitable quasi-difference. An equivalent characterization of this classification is given in terms of the number of linearly independent square summable solutions of the equation. The influence of off-diagonal coefficients on the classification is illustrated by two examples. In particular, two limit point criteria are established in terms of coefficients of the equation.Comment: 27 page

Real-space Formalism for the Euler Class and Fragile Topology in Quasicrystals and Amorphous Lattices

We propose a real-space formalism of the topological Euler class, which characterizes the fragile topology of two-dimensional systems with real wave functions. This real-space description is characterized by local Euler markers whose macroscopic average coincides with the Euler number, and it applies equally well to periodic and open boundary conditions for both crystals and noncrystalline systems. We validate this by diagnosing topological phase transitions in clean and disordered crystalline systems with the reality endowed by the space-time inversion symmetry $\mathcal{I}_{ST}$. Furthermore, we demonstrated the topological Euler phases in quasicrystals and even in amorphous lattices lacking any spatial symmetries. Our work not only provides a local characterization of the fragile topology but also significantly extends its territory beyond $\mathcal{I}_{ST}$-symmetric crystalline materials.Comment: 41 pages,8 figure

A MapReduce-based nearest neighbor approach for big-data-driven traffic flow prediction

In big-data-driven traffic flow prediction systems, the robustness of prediction performance depends on accuracy and timeliness. This paper presents a new MapReduce-based nearest neighbor (NN) approach for traffic flow prediction using correlation analysis (TFPC) on a Hadoop platform. In particular, we develop a real-time prediction system including two key modules, i.e., offline distributed training (ODT) and online parallel prediction (OPP). Moreover, we build a parallel k-nearest neighbor optimization classifier, which incorporates correlation information among traffic flows into the classification process. Finally, we propose a novel prediction calculation method, combining the current data observed in OPP and the classification results obtained from large-scale historical data in ODT, to generate traffic flow prediction in real time. The empirical study on real-world traffic flow big data using the leave-one-out cross validation method shows that TFPC significantly outperforms four state-of-the-art prediction approaches, i.e., autoregressive integrated moving average, Naïve Bayes, multilayer perceptron neural networks, and NN regression, in terms of accuracy, which can be improved 90.07% in the best case, with an average mean absolute percent error of 5.53%. In addition, it displays excellent speedup, scaleup, and sizeup

Privacy-Preserving Push-Pull Method for Decentralized Optimization via State Decomposition

Distributed optimization is manifesting great potential in multiple fields, e.g., machine learning, control, and resource allocation. Existing decentralized optimization algorithms require sharing explicit state information among the agents, which raises the risk of private information leakage. To ensure privacy security, combining information security mechanisms, such as differential privacy and homomorphic encryption, with traditional decentralized optimization algorithms is a commonly used means. However, this would either sacrifice optimization accuracy or incur heavy computational burden. To overcome these shortcomings, we develop a novel privacy-preserving decentralized optimization algorithm, called PPSD, that combines gradient tracking with a state decomposition mechanism. Specifically, each agent decomposes its state associated with the gradient into two substates. One substate is used for interaction with neighboring agents, and the other substate containing private information acts only on the first substate and thus is entirely agnostic to other agents. For the strongly convex and smooth objective functions, PPSD attains a $R$-linear convergence rate. Moreover, the algorithm can preserve the agents' private information from being leaked to honest-but-curious neighbors. Simulations further confirm the results

Projective Lag Synchronization of Delayed Neural Networks Using Intermittent Linear State Feedback

The problem of projective lag synchronization of coupled neural networks with time delay is investigated. By means of the Lyapunov stability theory, an intermittent controller is designed for achieving projective lag synchronization between two delayed neural networks systems. Numerical simulations on coupled Lu neural systems illustrate the effectiveness of the results